Image Feature Exploration

I manually picked 12 representatives from each category:

chicken_0116 chicken_0131 (so hard to distinguish) chicken_0181 chicken_0218 chicken_0264 chicken_0306 chicken_0383 chicken_0761 chicken_0646 chicken_0599 (only a leg) chicken_0891 (dark color) chicken_0999

dog_0010 dog_0021 (typical hair) dog_0023 dog_0033 dog_0095 dog_0146 (only a leg) dog_0147 dog_0342 (smooth surface) dog_0404 dog_0432 dog_0608 dog_0887

Some ideas: - Extract surface texture which stands out with curvy grains (? hard to realize) - Color RGB feature (success, 800 of them) - Color HSV feature

Here I pick 5 images (3 fried chickens and 2 dogs) to do an image exloration demo.

setwd("~/Google Drive/Columbia/5243 ADS/Project 3/Project3_poodleKFC_train/images")

The working directory was changed to /Users/mushroomvv/Google Drive/Columbia/5243 ADS/Project 3/Project3_poodleKFC_train/images inside a notebook chunk. The working directory will be reset when the chunk is finished running. Use the knitr root.dir option in the setup chunk to change the the working directory for notebook chunks.

library("EBImage")
i1.c <- readImage("chicken_0116.jpg") #chicken 1
i2.c <- readImage("chicken_0761.jpg") #chicken 2
i1.d <- readImage("dog_0021.jpg") #dog 1
i2.d <- readImage("dog_0608.jpg") #dog 2
i1 <- readImage("chicken_0131.jpg") # hard-to-distinguish chicken
filename=list(i1.c,i2.c,i1.d,i2.d,i1) #list all 5 images
options("EBImage.display"= "raster") #display in R rather than in browser
print(filename) #detailed info of all 5 images 

[[1]]
Image 
  colorMode    : Color 
  storage.mode : double 
  dim          : 310 400 3 
  frames.total : 3 
  frames.render: 1 

imageData(object)[1:5,1:6,1]
          [,1]      [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.3843137 0.3686275 0.3490196 0.3568627 0.4078431 0.4823529
[2,] 0.3921569 0.4117647 0.4274510 0.4313725 0.4431373 0.4745098
[3,] 0.3803922 0.4196078 0.4431373 0.4235294 0.4000000 0.4078431
[4,] 0.4000000 0.4156863 0.4117647 0.3764706 0.3529412 0.3725490
[5,] 0.4156863 0.4156863 0.4000000 0.3803922 0.3882353 0.4156863

[[2]]
Image 
  colorMode    : Color 
  storage.mode : double 
  dim          : 240 360 3 
  frames.total : 3 
  frames.render: 1 

imageData(object)[1:5,1:6,1]
          [,1]      [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.6470588 0.6509804 0.6549020 0.6549020 0.6549020 0.6549020
[2,] 0.6509804 0.6509804 0.6549020 0.6549020 0.6549020 0.6549020
[3,] 0.6509804 0.6509804 0.6549020 0.6509804 0.6509804 0.6509804
[4,] 0.6509804 0.6509804 0.6509804 0.6509804 0.6509804 0.6470588
[5,] 0.6509804 0.6509804 0.6509804 0.6509804 0.6470588 0.6470588

[[3]]
Image 
  colorMode    : Color 
  storage.mode : double 
  dim          : 1023 248 3 
  frames.total : 3 
  frames.render: 1 

imageData(object)[1:5,1:6,1]
     [,1] [,2] [,3] [,4] [,5] [,6]
[1,]    1    1    1    1    1    1
[2,]    1    1    1    1    1    1
[3,]    1    1    1    1    1    1
[4,]    1    1    1    1    1    1
[5,]    1    1    1    1    1    1

[[4]]
Image 
  colorMode    : Color 
  storage.mode : double 
  dim          : 450 160 3 
  frames.total : 3 
  frames.render: 1 

imageData(object)[1:5,1:6,1]
          [,1]      [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 0.8549020 0.8549020 0.8549020 0.8509804 0.8509804 0.8509804
[2,] 0.8549020 0.8509804 0.8509804 0.8431373 0.8431373 0.8352941
[3,] 0.8352941 0.8352941 0.8313725 0.8274510 0.8235294 0.8196078
[4,] 0.8039216 0.8039216 0.8078431 0.8078431 0.8078431 0.8117647
[5,] 0.7725490 0.7725490 0.7764706 0.7882353 0.7960784 0.8078431

[[5]]
Image 
  colorMode    : Color 
  storage.mode : double 
  dim          : 276 221 3 
  frames.total : 3 
  frames.render: 1 

imageData(object)[1:5,1:6,1]
          [,1]      [,2]      [,3]      [,4]      [,5]      [,6]
[1,] 1.0000000 0.9803922 0.8705882 0.7921569 0.7843137 0.8862745
[2,] 1.0000000 0.9450980 0.8901961 0.8745098 0.8823529 0.9372549
[3,] 0.9411765 0.9019608 0.9176471 0.9725490 0.9843137 0.9529412
[4,] 0.8823529 0.8784314 0.9529412 1.0000000 1.0000000 0.9215686
[5,] 0.8745098 0.8941176 0.9725490 1.0000000 0.9882353 0.8862745

for (i in 1:5){
  display(filename[[i]])   #display images
  hist(filename[[i]])      #RGB histograms
}

Change to Greyscale, blue and red channels

rr for (i in 1:5) { file.bl[[i]]=channel(filename[[i]], mode=‘asblue’) display(channel(filename[[i]], mode=‘asblue’)) }

rr for (i in 1:5) { file.red[[i]]=channel(filename[[i]], mode=‘asred’) display(channel(filename[[i]], mode=‘asred’)) }

Extract the texture by high contrast on blue channel,

which really shows difference to me between fried chicken and dogs.

rr file.contrast=list() for (i in 1:5){ file.contrast=file.bl[[i]]5 display(file.bl[[i]]5) }

Filtering: high pass

rr filter_high <- matrix(1, nc = 3, nr = 3) filter_high[2, 2] <- -8 file.hi=list() for (i in 1:5){ file.hi <- filter2(filename[[i]], filter_high) display(file.hi)# high pass }

Adaptive thresholding

Outline Analysis

rr for (i in 1:5){ oc[[i]] <- ocontour(bwlabel(img_leaf1[[i]])) plot(oc[[i]][[1]], type=‘l’);points(oc[[i]][[1]], col=2) }

Color Features from RGB

By discretizing the RGB values, we can get a set of color features. We subdivide the pixel values in each color channel into multiple bands of equal width. Then the counts of pixels in correponding bins consist of a set of color features that characterizes the color distribution of the image.

mat=list()
freq_rgb=list()
rgb_feature=list()
nR <- 10
nG <- 8
nB <- 10 # Caution: the bins should be consistent across all images!
rBin <- seq(0, 1, length.out=nR)
gBin <- seq(0, 1, length.out=nG)
bBin <- seq(0, 1, length.out=nB)
for (i in 1:5){
  mat[[i]] <- imageData(filename[[i]])
  freq_rgb[[i]] <- as.data.frame(table(factor(findInterval(mat[[i]][,,1], rBin), levels=1:nR), factor(findInterval(mat[[i]][,,2], gBin), levels=1:nG), factor(findInterval(mat[[i]][,,3], bBin), levels=1:nB)))
  rgb_feature[[i]] <- as.numeric(freq_rgb[[i]]$Freq)/(ncol(mat[[i]])*nrow(mat[[i]])) # normalization
}
str(rgb_feature) #each image has 800 features

List of 5
 $ : num [1:800] 0.01379 0.02122 0.02635 0.02134 0.00368 ...
 $ : num [1:800] 0.00301 0.01485 0.01819 0.02569 0.02503 ...
 $ : num [1:800] 7.88e-06 1.38e-03 8.54e-03 9.58e-03 3.15e-04 ...
 $ : num [1:800] 0.001236 0.005528 0.006125 0.000306 0 ...
 $ : num [1:800] 0.01459 0.01061 0.01105 0.00498 0.00152 ...

#The number of bins for each color channel (nR,nG,nB) is a tuning parameter for constructing color features. The total number of features is nR*nG*nB.

Color Features From HSV

The Hue, Saturation, Value (HSV) model of color is closer to human perception of color, and thus is eaiser to interpret than RGB model. Using the same discretization, color features can be extracted from the histogram of HSV values.

str(hsv_feature)

List of 5
 $ : num [1:360] 0.002895 0.001306 0.001065 0.001516 0.000565 ...
 $ : num [1:360] 0 0 0 0 0 ...
 $ : num [1:360] 0 0 0 0 0 0 0 0 0 0 ...
 $ : num [1:360] 1.86e-03 5.56e-05 2.78e-05 0.00 0.00 ...
 $ : num [1:360] 2.46e-04 6.56e-05 2.62e-04 3.11e-04 2.13e-04 ...